Method and device  for inputting a user&#39;s instructions based on movement sensing

ABSTRACT

Provided is a user instruction input device operating in a three-dimensional space. The user instruction input device includes a first sensor that senses the angular rate of the device centering on at least one axis, a second sensor that senses the acceleration of the device at least for one direction, and a processing unit that calculates a first rotation angle in a coordinate system independent of the attitude of the first device from the output value of the first sensor, calculates a second rotation angle in the coordinate system from the output value of the second sensor, and calculates the final attitude angle by combining the first rotation angle and the second rotation angle.

CROSS-REFERENCE TO RELATED APPLICATION

This application is based on and claims priority from Korean PatentApplication Nos. 10-2008-0113610, filed on Nov. 14, 2008, and10-2009-0027100, filed on Mar. 30, 2009, the disclosures of which areincorporated herein in their entireties by reference.

BACKGROUND OF THE INVENTION

1. Field of the Invention

Methods and apparatuses consistent with the present invention relate toa user instruction input device including a movement sensor. Moreparticularly, the present invention relates to a user instruction inputdevice capable of inputting user instructions naturally and accuratelybased on device movements in a three-dimensional space.

2. Description of the Related Art

As computer science develops, various devices that allow users to inputinformation in a computer device have been developed. One of suchdevices is called a user command input device. As a user manipulatessuch device components, position data corresponding to motion of theuser command input device are generated. Also, such position data areconverted into motions of a pointer image shown on the display. Hence,by moving the user command input device, the user may link the pointerimage with objects displayed on the display. Here, an object refers to auser interface that allows a certain action to be performed when a menu,a button or an image is selected. After that, the user can perform acertain command related with the corresponding object through aselection action such as pushing a certain button of the user commandinput device.

General personal computer users use operating systems with graphicaluser interfaces, such as Microsoft Windows and MAC OS X, to operatetheir computer. This is due to convenient mouse functions and variousgraphic functions that are not supported in console based operatingsystems such as DOS (Disk Operating System) system and some UNIXversions, and users can simply input commands through a mouse dragging,scrolling or a click without a keyboard input.

On the other hand, various commands are inputted using a key padprepared on a remote control device in an image display device thatcannot use a keyboard or a mouse that is used in a personal computer,such as a digital TV, a set-top box, a game machine. Such a key padinput method has been mainly used because such devices are not fixed ona certain position for the operation of the device unlike a personalcomputer, and operation is necessary in an open space such as a livingroom, so it is difficult to use an input means fixed on a plane such asa keyboard or mouse.

Considering such problems, three-dimensional user command input deviceswith a motion sensor such as a gyroscope and an accelerometer arerecently being developed. By moving a three-dimensional user commandinput device, a user can move a pointer image on the correspondingdisplay in a desired direction and at a desired speed, and by pushing acertain button on the user command input device, the user can select andexecute a desired action.

However, unlike a technology that inputs user commands through actionson a fixed two-dimensional plane as in a mouse, it is not easy totransmit natural and accurate motions in a user command input devicethat moves a pointer or a certain object (e.g., a game unit) through anarbitrary action in a three-dimensional space. It is because motionsthat have not been intended by the user may be transmitted depending onthe pose, orientation or distance toward the device.

In fact, inventions about measuring three-dimensional motions usingaccelerometers and angular rate sensors have been made since 1980s. Thepresent invention does not simply intend to implement an input deviceusing an accelerometer and an angular rate sensor, but intends toimplement an input device that naturally fits the user's intention witha compact system (i.e., a system that uses a small amount ofoperations).

SUMMARY OF THE INVENTION

An objective of the present invention is to input user instructions morenaturally and accurately in a device that inputs user instructionsthrough arbitrary movements in a three-dimensional space.

The present invention will not be limited to the technical objectivesdescribed above. Other objectives not described herein will be moredefinitely understood by those in the art from the following detaileddescription.

According to an exemplary embodiment of the present invention, there isprovided a user instruction input device operating in athree-dimensional space including a first sensor that senses the angularrate of the device centering on at least one axis, a second sensor thatsenses the acceleration of the device at least for one direction, and aprocessing unit that calculates a first rotation angle in a coordinatesystem independent of the attitude of the first device from the outputvalue of the first sensor, calculates a second rotation angle in thecoordinate system from the output value of the second sensor, andcalculates the final attitude angle by combining the first rotationangle and the second rotation angle.

According to an exemplary embodiment of the present invention, there isprovided a method of inputting a user instruction using a userinstruction input device operating in a three-dimensional spaceincluding sensing the angular rate of the device on at least one axis,sensing the acceleration of the device for at least one direction,calculating a first rotation angle in a coordinate system independent ofthe attitude of the device using the output value of the first sensor,calculating a second rotation angle in the coordinate system using theoutput value of the second sensor, calculating the final attitude angleby combining the first rotation angle and the second rotation angle, andoutputting a position variation corresponding to the variation of thecalculated final attitude angle.

BRIEF DESCRIPTION OF THE DRAWINGS

The above and other features and advantages of the present inventionwill become apparent by describing in detail preferred embodimentsthereof with reference to the attached drawings in which:

FIG. 1 illustrates a three-axis rotation angle defined in a certainframe.

FIG. 2 is a block diagram illustrating a user instruction input deviceaccording to an exemplary embodiment of the present invention.

FIG. 3 illustrates a method of measuring a roll value using anaccelerometer according to an exemplary embodiment of the presentinvention.

FIG. 4 illustrates a method of calculating a yaw value using the pitchand roll values in the method of measuring a yaw angle according to anexemplary embodiment of the present invention.

FIGS. 5 to 7 show the case when ω_(x) is used and the case when ω_(x) isnot used in equation 5.

FIG. 8 is flowchart estimating movements of the device using a movementestimation function.

FIG. 9 shows a bell-shaped curve as a movement estimation function.

FIG. 10 shows an exponential function used as a movement estimationfunction.

FIGS. 11A, 11B and 12 show a mapping function according to an exemplaryembodiment of the present invention.

FIGS. 13 and 14 show examples of applying a mapping scale regardingpitch-direction rotations.

FIGS. 16 and 17 show examples of applying a mapping scale regardingyaw-direction rotations.

FIG. 18 shows the agreement between the starting point of a body frameand the starting point of a navigation frame.

FIG. 19 shows the disagreement between the starting point of a bodyframe and the starting point of a navigation frame.

DETAILED DESCRIPTION OF THE INVENTION

Exemplary embodiments of the present invention will be described indetail with reference to the accompanying drawings.

Advantages and features of the present invention and methods ofaccomplishing the same may be understood more readily by reference tothe following detailed description of the exemplary embodiments and theaccompanying drawings. The present invention may, however, be embodiedin many different forms and should not be construed as being limited tothe embodiments set forth herein. Rather, these embodiments are providedso that this disclosure will be thorough and complete and will fullyconvey the concept of the invention to those skilled in the art, and thepresent invention will only be defined by the appended claims.

In the present invention, a user instruction input device refers to aninterface device that makes various contents intuitively available byreceiving the input of the user's movements. The device makes theinformation obtained through the user's movements correspond toinformation that is necessary for various information devices or variousservices. Some examples of such devices are a three-dimensional spacemouse, an IPTV (Internet protocol TV) remote control and a game inputdevice.

FIG. 1 illustrates a three-axis rotation direction defined in a certainframe (coordinate system). In a certain frame consisting of x, y and zaxes, a pitch (θ) refers to a rotation in y-axis direction, a roll (φ)refers to a rotation in x-axis direction, and a yaw (ψ) refer to arotation in z-axis direction. Whether a rotation is positive (+) ornegative (−) is determined by a right-handed coordinate. The presentinvention mentions two frames of a navigation frame and a body frame.The navigation frame is fixed on space and refers to a standardcoordinate system consisting of three axes of X_(N), Y_(N) and Z_(N).That is, the navigation frame is independent of the attitude of adevice. Further, the body frame refers to a relative coordinate systemthat exists on an object placed in three-dimensional space and consistsof three axes of X_(B), Y_(B) and Z_(B). In FIG. 1, x-direction refersto a standard direction where the user instruction input device istoward. That is, it is a roll direction if a rotation is made based onthe axis toward the standard direction.

FIG. 2 is a block diagram illustrating a user instruction input device100 according to an exemplary embodiment of the present invention. Theuser instruction input device 100 can control an object on at least onedisplay device (not shown). The object may be a pointer on a display ormay be an object while playing a game. Also the display device may beinstalled on a separately fixed position, but it may also be integratedwith the user instruction input device 100 (e.g., a portable gamemachine).

As a more specific example, the user instruction input device 100 mayinclude an angular rate sensor 110, an acceleration sensor 120, afiltering unit 130, a processing unit 190 and an output unit 180.Further, the processing unit 190 may include a first operation unit 140,a second operation unit 150, a attitude angle measuring unit 160 and avariation mapping unit 170.

The angular rate sensor 110 senses an angular rate at which the device100 rotates on the body frame, and provides a sampled output value(digital value). A gyroscope can be used as angular rate sensor 110, andvarious types of gyroscopes such as a mechanical type, a fluid type, anoptical type and a piezoelectric type. Specifically, the angular ratesensor 110 can obtain rotational angular rates for two axes (axes on thebody frame) that cross at right angles, e.g., a rotational angular rate(ω_(x), ω_(y), ω_(z)) on x-axis, y-axis and z-axis of the body frame.

The acceleration sensor 120 senses the acceleration of the device 100 onthe body frame and provides a sampled output value (digital value). Theacceleration sensor 120 can be a piezoelectric type or a moving coiltype. Specifically, the angular rate sensor 110 calculates the straightline acceleration (f_(x), f_(y), f_(z)) for three axes that cross atright angles (axes on the body frame).

The filtering unit 130 may consist of a low pass filter, a high passfilter, a offset filter or a scaling filter depending on the usage ofthe device 100, compensates for the error after receiving the output ofthe angular sensor 110 and the output of the acceleration sensor 120.The filtering unit 130 provides the error-compensated rotational angularrate (ω_(x), ω_(y), ω_(z)) to the first operation unit 140 and providesthe error-compensated acceleration speed (f_(x), f_(y), f_(z)) to thesecond operation unit 150.

The second operation unit calculates the roll, pitch and yaw of thenavigation frame (φ_(XL), θ_(XL), ψ_(XL)) (the second rotational angle)using the straight line angular rate (f_(x), f_(y), f_(z)) provided fromthe filtering unit 130. A specific example of the calculation is shownin the following.

$\begin{matrix}{{\varphi_{XL} = {{arc}\; \tan \; 2\left( \frac{f_{y}}{f_{z}} \right)}}{\theta_{XL} = {\arctan \; 2\left( \frac{- f_{x}}{\sqrt{f_{y}^{2} + f_{z}^{2}}} \right)}}{\psi_{XL} = \frac{\varphi_{XL}}{\sin \left( \theta_{XL} \right)}}} & {{Equation}\mspace{14mu} 1}\end{matrix}$

Generally, the roll and the pitch (φ_(XL), θ_(XL)) can be simplycalculated using the acceleration, but it is not easy to get the yaw.The yaw (ψ_(XL)) calculated from the acceleration in the presentinvention is a pseudo yaw and can be explained with reference to FIGS. 3and 4.

FIG. 3 illustrates a method of measuring the roll value using theaccelerometer according to an exemplary embodiment of the presentinvention. Assuming that the direction acceleration speed is f_(y), andthe pitch has already been generated, the vertical element ofacceleration of gravity becomes g·cos θ in (b) of FIG. 3. Hence, theequation of the roll is as follows.

$\begin{matrix}{\varphi = {\arcsin\left( \frac{f_{y}}{{g \cdot \cos}\; \theta} \right)}} & {{Equation}\mspace{14mu} 2}\end{matrix}$

The method of measuring the roll value using the accelerometer can becalculated in various methods.

FIG. 4 illustrates a method of calculating a yaw value using the pitchand the roll values in the method of measuring the yaw angle accordingto an exemplary embodiment of the present invention. (a) of FIG. 4 showsa vector direction of the roll and yaw angular rate based on thegeneration of the pitch. In (a) of FIG. 4, ω_(y) represents the rollangular rate vector, and ω_(z) represents the yaw angular rate vector.The yaw angular rate vector is not the actual yaw vector, but theprojected vector of the actual yaw vector.

(b) of FIG. 4 illustrates (a) of FIG. 4 from the side. In (b) of FIG. 4,assuming that the time is “t,” the following equation 3 is establishedamong the roll angular rate vector ω_(y), the yaw angular rate vectorω_(z) and the pitch (θ).

$\begin{matrix}{{\sin \; {\theta (t)}} = \frac{\omega_{y}(t)}{\omega_{z}(t)}} & {{Equation}\mspace{14mu} 3}\end{matrix}$

From the above equation 3, the yaw (Ψ) can be approximated as shown inthe equation 4.

$\begin{matrix}{\psi = {{\int{\omega_{z}{t}}} = {{\int\frac{\omega_{y}(t)}{\sin \; {\theta (t)}}}\overset{\sim}{=}{{\frac{1}{\sin \; \theta}{\int{\omega_{y}(t)}}} = \frac{\varphi}{\sin \; \theta}}}}} & {{Equation}\mspace{14mu} 4}\end{matrix}$

Equation 4 cannot be applied if pitch θ is close to 0° or 90°, so acertain restriction should be given at such angles. The actuallymeasured yaw values in each condition using equation 4 are shown in thefollowing.

Yaw (Ψ) measured Yaw (Ψ) measured at (−) pitch at (+) pitch Pitch (θ)roll Φ = −80 roll Φ = +80 roll Φ = −80 roll Φ = +80 10 Strange Strange−414 383 movement movement 20 −222 243.5 −221 241.5 40 −120 125 −122 12760 −89 91.5 −92 94.5 80 −72.5 77 −83 84.5

As shown in the above table, as the roll change, the yaw value changes,so the scale elements can be used to reduce such differences.Consequently, in the situation when there is a pitch value and both theroll value and the yaw value change, it is possible to calculate theapproximate value for the yaw.

The first operation unit 140 calculates the rotation angle (φ_(G),θ_(G), ψ_(G)) (the first rotation angle) of three axes in the navigationframe using angular rate (ω_(x), ω_(y), ω_(z)) value provided from thefiltering value 130. The specific equation using the Euler angle isshown in equation 5. Equation 5 has a form of a differential equationfor the rotation angle (φ_(G), θ_(G), ψ_(G)) in the navigation frame.

$\begin{matrix}{{{\overset{\cdot}{\varphi}}_{G} = {{\left( {{\omega_{y}\sin \; \varphi_{G}} + {\omega_{z}\cos \; \varphi_{G}}} \right)\tan \; \theta_{G}} + \omega_{x}}}{{\overset{\cdot}{\theta}}_{G} = \left( {{\omega_{y}\cos \; \varphi_{G}} - {\omega_{z}\sin \; \varphi_{G}}} \right)}{{\overset{\cdot}{\psi}}_{G} = \frac{{\omega_{y}\sin \; \varphi_{G}} + {\omega_{z}\cos \; \varphi_{G}}}{\cos \; \theta_{G}}}} & {{Equation}\mspace{14mu} 5}\end{matrix}$

Generally, the attitude angle in the navigation frame can be obtainedusing three angular rates. The desirable embodiment of the presentinvention calculates angular rates of three axes in the navigation frameusing only two angular rates (ω_(y), ω_(z)), and here ω_(x) of equation5 is a problem.

General hand and arm movements of a person are often based on a singleaxis in a three-axis coordinate system. Some such examples are rotationon Z-axis and rotation in Y-axis direction. Also, even though there aretwo or more composite movements at the same time, there is a tendencythat when X-axis and Y-axis rotations of the body frame and X-axis andZ-axis rotations of the body frame occur at the same time, movements inX-axis direction becomes relatively smaller than those in Y-axis andZ-axis directions.

Hence, in equation 5, ω_(x) in X-axis direction becomes relativelysmaller than that in Y-axis direction or Z-axis direction, so ω_(x) maybe disregarded, but the calculation is not done accurately. However, ifthe roll (φ_(XL)) information calculated by the accelerometer isappropriately used, a performance, which is similar to that when theangular rate of ω_(x) is used, can be secured.

FIGS. 5-7 are the result of comparing the case of using ω_(x) (dottedline) and the case of not using ω_(x) (solid line). X-axis refers to thenumber of unit samples, and Y-axis refers to the angle. Also, the rightside graph of each drawing is an extended view of a certain sectionindicated by a circle in the left side graph. When ω_(x) is removed,there is an 2°˜4° difference and a delay of 2 to 10 samples than whenω_(x) is not removed. However, if the calculation period is more than100 Hz, the angular difference and the delay are not easilydistinguishable by the user, so there is no significant difference inthe result between when using the angular rate for two axes (Y-axisangular rate and Z-axis angular rate of the body frame) and when usingthe angular rate for three axes.

Based on such an experiment result, equation 5 can be transformed to thefollowing equation 6.

$\begin{matrix}{{{\overset{\cdot}{\varphi}}_{G} = {\left( {{\omega_{y}\sin \; \varphi_{G}} + {\omega_{z}\cos \; \varphi_{G}}} \right)\tan \; \theta_{G}}}{{\overset{\cdot}{\theta}}_{G} = \left( {{\omega_{y}\cos \; \varphi_{G}} - {\omega_{z}\sin \; \varphi_{G}}} \right)}{{\overset{\cdot}{\psi}}_{G} = \frac{{\omega_{y}\sin \; \varphi_{G}} + {\omega_{z}\cos \; \varphi_{G}}}{\cos \; \theta_{G}}}} & {{Equation}\mspace{14mu} 6}\end{matrix}$

Further, if a pattern when a person grabs the input device and moves isrecognized in advance and is utilized, even ω_(y) can be removed. Here,as in equation 1, errors generated by not using ω_(y), ω_(z) can beovercome by using (φ_(XL), θ_(XL), ψ_(XL)) calculated using the outputof the accelerometer.

Equations 5 and 6 illustrate the calculation of the rotation angle(φ_(G), θ_(G), ψ_(G)) of three axes in the navigation frame from theangular rate (ω_(y), ω_(z)) based on the Euler angle representation, butthe calculation may be performed based on the more involved quaternionangle representation instead of Euler angle.

Referring to FIG. 2, the attitude angle measuring unit 160 calculatesthe weighted average of three rotation angles (φ_(XL), θ_(XL), ψ_(XL))on the navigation frame obtained from the second operation unit 150 andthe three rotation angles (φ_(G), θ_(G), ψ_(G)) on the navigation frameobtained from the first operation unit 140, and calculates the attitudeangle (φ, θ, ψ) in the navigation frame. The specific weighted averagecan be calculated according to the following equation 7.

φ=α₁·φ_(XL)+(1−α₁)·φ_(G)

θ=α₂·θ_(XL)+(1−α₂)·θ_(G)

ψ=α₃·ψ_(XL)+(1−α₃)·ψ_(G)  Equation 7

Here, α₁, α₂ and α₃ are weights for φ, θ and ψ, respectively. The aboveprocess to calculate the attitude angle in the navigation is just anexample, but various other ways can be used to calculate the attitudeangle. For example, ψ_(XL) used in equation 1 can be calculated using amagnetic sensor or an image sensor. According to the magnetic sensor orthe image sensor, because a reentering angle in the navigation frame canbe directly calculated, a transformation process as in equation 1 usedin the acceleration sensor is not necessary.

If ψ_(XL) is not calculated from the third formula of equation 1, i.e.,θ_(XL)=0, by setting α₃ to 1, ψ can be calculated only by ψ_(G) withoutusing ψ_(XL).

However, in order to calculate the final attitude angle (φ, θ, ψ) moreaccurately, α₁, α₂ and α₃ need to be adaptively determined rather thanarbitrarily determined. For this, the “attitude angle measuring” angle160 can use a movement estimation function.

The movement estimation function refers to a function that detectsdetailed movements by normalizing movements to the range between 0 and 1based on data using the outputs of the angular rate sensor and theacceleration sensor. As an example, in the case where movements of thedetected device 100 are hardly noticeable, the value of the accelerationsensor 120 is more reliable, so the mapping is done so that α_(n)=1, andin the case where the movements are the maximum state, the value of theangular rate sensor 110 is more reliable, so the mapping is done so thatα_(n)=0. In the case where the movements are between the hardlynoticeable state and the maximum state, the mapping should be done withan appropriate value.

FIG. 8 is a flowchart for estimating movements of a device 100 using amovement estimation function. A bell shape curve or an exponentialfunction may be used as a movement estimation function. The bell shapecurve, as a curve in the form shown in FIG. 9, may be expressed byGaussian function, a raised cosine function, etc. Further, theexponential function has the form shown in FIG. 10, and is, e.g.,y=e^(−|x|). In FIGS. 9 and 10, x-axis represents the size of movementsof the device 100, and y-axis represents the weight in equation 7, i.e.,α_(n). Functions of FIGS. 9 and 10 commonly have the peak value in thecenter, and have the form approaching 0 as they go right or left.

By using such a movement estimation function, detailed movements such asstoppage, minute movements, slow movements and fast movements can bedetected in addition to detecting whether the device 100 has stopped.Further, by measuring such movements, the basis for removing unintendedmovements (e.g., a cursor movement by a trembling of a hand) can beprovided. Further, it is possible to adjust the scale of a mappingfunction according to the size of movements, which will be explainedlater.

Referring to FIG. 2 once again, in the variation mapping unit 170 theattitude angle (φ, θ, ψ) itself obtained in the attitude angle measuringunit 160 can be used in controlling the object of a display device. Forexample, in a flying object in a flight simulation, each axis directionrotation can be controlled by the attitude angle. Further, a pointerexisting on the two-dimensional screen of the display device needs to bemapped with the variation (displacement) of the attitude angle (φ, θ, ψ)of such a navigation coordinate system. As an example, in order toexpress natural movements of a pointer control device, horizontalmovements (Δx) on the display screen need to correspond to the variation(Δψ) of the yaw, and vertical movements need to correspond to thevariance (Δθ) of the pitch.

Such a relation can be expressed as the following equation 8.

Δψ=ψ_(k)−ψ_(k−1)

Δθ=θ_(k)−θ_(k−1)  Equation 8

Here, “k” is an index that indicates a certain sampling time point.

In order to define such a correspondence relation, the present inventionintroduces a mapping function. FIGS. 11 and 12 illustrate examples ofvarious mapping functions and show correspondence between horizontalmovements (Δx) and variation of the yaw (Δψ). The same is applied to thecorrespondence between vertical movements (Δy) and the variation of thepitch (Δθ).

FIGS. 11A and 11B show examples of simple mapping functions. Here, FIG.11A shows a floor function having the form of ƒ(t)=└kt┘, and FIG. 11Bshows a ceiling function having the form of ƒ(t)=┌kt┐. Such mappingfunctions can make the variation of the yaw or the pitch correspond tohorizontal movements or vertical movements on the display by a simplerelation. Here, the input value of functions can be reduced or removeddepending on the change of “K”, and an input value more than a certainvalue can be converted into a proportional output value. However, themapping functions such as FIGS. 11A and 11B have a disadvantage thatoutput values cannot be appropriately limited for excessive input valuesmore than a certain value. That is, the mapping function of FIGS. 11Aand 11B do not have the function of the limitation area among thefunctions of the depression area, the scaling area and the limitationarea included in the mapping function.

The mapping function of FIG. 12 is divided into three areas, which ismore complicated than the functions of FIGS. 11A and 11B. The first area(part 1) is a depression area that does the mapping after reducing theinput value of the function. The second area (part 2) is a scaling areathat maps the input value to a roughly proportional output value.Finally, the third area is a limitation area that limits the outputvalue for the input more than a certain value.

An example of mapping functions of FIG. 12 is Sigmoid function such as

${f(t)} = {\frac{1}{1 + ^{- t}}.}$

Here, there are both positive and negative movement directions, so theSigmoid function is symmetrical on the starting point of thecoordinates. That is, the mapping function of FIG. 12 is made ofcombination of the same two Sigmoid functions. It is desirable for avalue determined as a HID (human interface device) mouse standard to beused as the position variation obtained by the mapping.

The meaning of three areas of FIG. 12 is explained in more detailed inthe following. The first depression area is an area where the user'spointing movements are minute. In this area, the variation of theattitude angle is not mapped with the position variation by 1:1, but themapping is done by reducing the position variation, which provides thefunction that removes unintended movements such as a trembling of auser's hand. However, in applications that need implementation of minutemovements, such movements can be made to be expressed by raising up thedepression area.

The second scaling area is an area that proportionally maps actual usermovements to position information on the display device, and the mappingcan be done by ±128 integer values according to HID mouse standard.

The third limitation region is an area that limits position variations(Δx, Δy) when the user's movements are relatively large. In FIG. 12, themapping function has a symmetrical form for the starting point, sosymmetrical outputs can be drawn in positive and negative directions forthe user's movements.

Further, such a mapping function can be scaled in various ways in linewith the user's movement pattern, which is explained in the following.

The user most comfortably grabs the device 100 when the pitch angle is5°-10° as shown in FIG. 13. However, if the pitch angle becomes 30°-50°,the control of the pointer 10 toward Y-axis positive direction becomesrelatively more difficult than the control of the pointer 10 toward theY-axis negative direction. Likewise, the difficulty due to thelimitation of the wrist movement in implementing the positioninformation is called wrist jamming. The same wrist jamming can existfor each direction of the yaw (ψ).

FIG. 14 shows an example of such a wrist jamming. The device 100 israised up about more than 40°, but the position of the pointer 10 on thedisplay screen is positioned near the lower side of the display device.In this case, it becomes difficult for the user to move in the upperdirection of the display device due to the limitation of the wristmovement.

Hence, in the device 100 according to an exemplary embodiment of thepresent invention, if the pitch angle exceeds a certain angle, the scaleis increased for the movement that raises up in the positive pitch angledirection, the scale is decreased for the movement that lowers in thenegative pitch angle direction, by which the user can continually pointin the most comfortable state of 5° to 10° pitch angle.

The variation mapping unit 170 of the device 100 can simply do suchoperations by adjusting the mapping scale applied to the entire mappingfunctions. The adjustment of the mapping scale can be applied in variousmanners according to each set of pitch information, and the same can beapplied even when the pitch angle is negative.

The technique used in the pitch (θ) angle can be applied for the yaw (ψ)angle. In the case where the user operates the device 100, the movementis usually done between ±45° in the yaw rotation direction as shown inFIG. 16. However, if the yaw angle of the device 100 exceeds the range,the user feels difficulty in pointing due to the limitation of the wristmovement.

Hence, in the device 100 according to an exemplary embodiment of thepresent invention, when the yaw angle exceeds a certain angle in thepositive direction, the device increases the scale for the movement thatrotates in the positive yaw angle direction and decrease the scale forthe movement in the negative yaw angle direction. The same can beapplied in the state where the yaw angle of the device 100 exceeds acertain angle in the negative direction. Hence, the user's pointing canbe continued between ±45° yaw angles when the user feels mostcomfortable. The variation mapping unit 170 of the device 100 can simplyperform such an operation by adjusting the mapping scale applied to theentire mapping function.

Further, in the case where the device 100 is raised up by 90° in thepitch direction, the position variations (Δx, Δy) on the display screencan be made to be limited. For example, the variation mapping unit 170can be made for the mapped position variations (Δx, Δy) not to begenerated by setting Δψ and Δθ to “0” when θ is close to 90°. It isbecause there can be pointer movements unintended by the user as asingular point is generated in the third formula (a formula about ψ) ofequation 6 when the pitch angle becomes 90°.

Such a problem can be solved if the calculation is done using aquaternion angle representation, instead of Euler angles, but the amountof calculation increases, so each method has advantages anddisadvantages. However, the user rarely operates the pointer whilegrabbing the device perpendicularly, and even if it happens, it can beresolved by limiting the position variation as described above.

Referring to FIG. 2, the output unit 180 wirelessly transmits theattitude angles (φ, θ, ψ) provided from the attitude angle measuringunit 160 and the position variation (Δx, Δy) provided from the variationmapping unit to the display device depending on the type of theapplication. If the display device is integrally implemented with thedevice 100, the data can be transmitted to the main processor. Thewireless transmission can be done through Bluetooth communication,infrared communication, IEEE 802.11 wireless LAN standard, IEEE802.15.3. wireless LAN standard, etc.

Each block of FIG. 2 can be implemented by a task, a class, asub-routine, a process, an object, an execution thread, software such asa program, hardware such as FPGA (field-programmable gate array) andASIC (application-specific integrated circuit) or a combination of thesoftware and the hardware performed in a predetermined area of a memory.Also, each block may represent a portion of code, a segment or a moduleincluding one or more executable instructions to execute specifiedlogical functions. Also, in some alternative examples, theabove-mentioned functions can occur regardless of the order. Forexample, two consecutive blocks may be practically performed at the sametime, or may even be performed in a reverse order depending to theirfunctions.

A user instruction input device according to an exemplary embodiment ofthe present invention has practical advantages as follows.

1. Simplified System

The user instruction input device 100 only uses the conversion for therotation information between two coordinate systems (frame) in theprocess of converting the body information into the navigation frame.That is, the starting point of the body frame and the starting point ofthe navigation frame are kept in the same state as in FIG. 18. Likewise,when the coordinate system is converted, the change of the startingpoints (O₂-O₁ of FIG. 19) between two coordinate systems is omitted, sothe amount of calculation can be reduced.

Also, as in equation 7, by using the concept of the weighted average,the movement estimation function and the mapping function, thecomplicated operations for sampling data, which is necessary for amodel-based filtering such as linear filtering, Kalman filtering, Kalmansmoothing, extended Kalman filtering, state-space estimation andexpectation-maximization, and the following initialization time are notrequired.

Further, the movement estimation function and the mapping function use asimplified form of 1:1 correspondence function unlike the matrixoperation that occupies many resources. Hence, as such a simplifiedfunction is used, the operation time and resources are significantlyreduced.

2. Attitude Angle in Navigation Frame

The user instruction input device 100 measures the rotation andacceleration of the body frame to calculate attitude information of thenavigation frame. The position information (e.g., the movement of apointer) is controlled through the obtained attitude information of thenavigation frame, so the position information can be implementedregardless of the slant of the body frame. As stated above using theroll, pitch and yaw information (attitude angle information) in thenavigation frame, various applications such as implementation ofmovements according to the user pattern, intuitive movementimplementation and the control of the state of the display device arepossible. Also, in the product design, there is no limitation on therotation direction of the body frame, various designs regarding theouter appearance of the user instruction input device 100 are possible.

3. Movement Detection

In the existing technology, a simple stoppage is determined, but in theuser instruction input device 100, the various movements such as thestoppage, minute movements, slow movements and fast movements can bedetected, and the basis for more improved movements is provided throughsuch various forms of movement detections. Also, the movement estimationfunction used in detecting movements makes the input correspond to theoutput by 1:1, thereby not occupying many resources.

4. Mapping

The mapping function consisting of the depression area, the scaling areaand the limitation area provides more detailed and intuitive movements.Movements such as a trembling of a hand are removed through thedepression area, and minute movements can be implemented through thescaling area, and the limitation of excessive movements is possiblethrough the limitation area. In such a mapping function, it is possibleto only take values of desired areas.

Also, the mapping function can convert a floating point value into anormalized digital value, and through the digitalization, advantages ofdigital signals such as reduction of noise and reduction of transmitteddata amount are provided. Also, the mapping function used here does noterequire a separate initialization time and data sampling as in amodel-based filtering, and is a simplified function where the inputcorresponds to the output by 1:1.

5. Stable and Accurate Movement Implementation Using Weighted Averages

In the user instruction input device 100, the weighted average iscalculated based on each set of information implemented by the output ofthe angular rate sensor and the output of the accelerometer, so morestable and accurate movements can be implemented. If only the angularrate is used, accumulated errors by the bias change are generated in theprocess of implementing angles by integrating angular rates, therebygenerating divergence of angular rates, and there technologies toresolve such a problem are known. Some examples of such technologies area method of estimating a bias through Kalman filter, a method of using adigital filter through frequency analysis and a method of estimating abias by analyzing given time and critical values. However, all suchexisting technologies excessively consume system resources and require alarge amount of operation.

In contrast, in the case of the user instruction input device 100, theconcept of the weighted average is applied using the angular rate sensorand the accelerometer, so the divergence of the angle can be simplylimited by the accelerometer, by which the model or the filter used inestimating the bias of the angular rate can be simplified, and theaccuracy of the attitude angle measurement can be improved. Through thecalculated attitude angle, more stable and accurate movements can beimplemented.

Also, in the user instruction input device 100, if the movement isdetected by the movement estimation function, the user-intended variousmovements can be expressed by changing each weight (α₁, α₂, α₃ ofequation 7) according to the detected movement. That is, appropriatemovements harmonized with the situation can be implemented.

6. Reducing System Initialization Time

According to a simplified system of the user instruction input device100, the initialization time of the device can be reduced. Arepresentative example is the concept of a weighted average. If an angleis implemented only by an angular rate sensor, a separate filteringtechnique such as Kalman filtering is required to minimize accumulatederrors. In the case of the Kalman filtering, the initialization step forthe initial bias estimation or bias setting is essentially required.

In contrast, the user instruction command device 100 can calculate theaccurate angle without such an initialization step by implementing eachset of information using the weighted average with the output of theaccelerometer along with the accelerometer. That is, the accuracy of theattitude angle measurement can be improved with the system issimplified.

Also, unlike the model-based filtering, the mapping function centers on1:1 correspondence, so it contributes to initialization time reductionto some extent.

7. Reducing Power Consumption

According to a simplified system of the user instruction input device100, the consumption power can be reduced based on the reduction of theinitialization time and operation amount. As the initialization time ofthe system is reduced, there can be simplified operation modes such asoperation mode, power-down mode and power-off mode.

The existing technologies needed temporary steps such as a stand-by modethat stabilizes the system for entering the operation mode. However, theuser instruction input device 100 does not need a temporary step such asa stand-by mode, so the power supplied to a certain element can beselectively turned off. Hence, if the on-off of the device power getseasy, the consumption power can be more reduced.

It should be understood by those of ordinary skill in the art thatvarious replacements, modifications and changes may be made in the formand details without departing from the spirit and scope of the presentinvention as defined by the following claims. Therefore, it is to beappreciated that the above described embodiments are for purposes ofillustration only and are not to be construed as limitations of theinvention.

1. A user instruction input device operating in a three-dimensionalspace, the device comprising: a first sensor that senses the angularrate of the device centering on at least one axis; a second sensor thatsenses the acceleration of the device at least for one direction; and aprocessing unit that calculates a first rotation angle in a coordinatesystem independent of the attitude of the first device from the outputvalue of the first sensor, calculates a second rotation angle in thecoordinate system from the output value of the second sensor, andcalculates the final attitude angle by combining the first rotationangle and the second rotation angle.
 2. The device of claim 1, whereinthe first sensor is an angular rate sensor that senses rotation of thedevice on the first axis and rotation of the device on the second axisperpendicular to the first axis.
 3. The device of claim 1, wherein thecalculated first rotation angle and the calculated second rotation angleare combined by the weighted average.
 4. The device of claim 3, whereinthe weight is estimated by a predetermined movement estimation function,the movement estimation function sets the weight to reflect only thesecond rotation angle to the final attitude angle if there is nomovement in the device, the function sets the weight to reflect only thefirst rotation angle if the movement of the device is at the maximumlevel, and the function sets the weight to between the value at nomovement and the value at the maximum movement if the movement of thedevice is between no movement and the maximum movement.
 5. The device ofclaim 1, wherein the first rotation angle is calculated from adifferential equation of the combination of the output of the firstsensor and the first rotation angle
 6. The device of claim 1, whereinthe yaw of the second rotation angle is calculated from the combinationof the roll and the pitch of the second rotation angle.
 7. The device ofclaim 1, wherein the final attitude angle includes at least a yaw and apitch.
 8. The device of claim 7, wherein the variation of the yawcorresponds to the horizontal variation of the pointer on the display,and the variation of the pitch corresponds to the vertical variation ofthe pointer on the display.
 9. The device of claim 8, wherein thevariation of the yaw and the horizontal variation, and the variation ofthe pitch and the y-axis variation are mapped by a predetermined mappingfunction.
 10. The device of claim 9, wherein the mapping functionincludes at least two of three areas, which are a mapping depressionarea that outputs by reducing or removing the input value of thefunction, a scaling area that maps the input value of the function to aroughly proportional output value, and a limitation area that limits theoutput value for the input value more than a certain value.
 11. A methodof inputting a user instruction using a user instruction input deviceoperating in a three-dimensional space, the method comprising: sensingthe angular rate of the device on at least one axis; sensing theacceleration of the device for at least one direction; calculating afirst rotation angle in a coordinate system independent of the attitudeof the device using the output value of the first sensor; calculating asecond rotation angle in the coordinate system using the output value ofthe second sensor; calculating the final attitude angle by combining thefirst rotation angle and the second rotation angle; and outputting aposition variation corresponding to the variation of the calculatedfinal attitude angle.
 12. A user instruction input device operating in athree-dimensional space, the device comprising: a first sensor thatsenses the angular rate of the device on at least one axis; a secondsensor that senses the rotation angle of the device; and a processingunit that calculates the rotation angle in a coordinate systemindependent of the attitude of the device using the output value of thefirst sensor, and calculates the final attitude angle by combining thesensed rotation angle and the calculated rotation angle.